Understanding Physician's Perspectives on AI in Health Care: Protocol for a Sequential Multiple Assignment Randomized Vignette Study.

BACKGROUND: As the availability and performance of artificial intelligence (AI)-based clinical decision support (CDS) systems improve, physicians and other care providers poised to be on the front lines will be increasingly tasked with using these tools in patient care and incorporating their outputs into clinical decision-making processes. Vignette studies provide a means to explore emerging hypotheses regarding how context-specific factors, such as clinical risk, the amount of information provided about the AI, and the AI result, may impact physician acceptance and use of AI-based CDS tools. To best anticipate how such factors influence the decision-making of frontline physicians in clinical scenarios involving AI decision-support tools, hypothesis-driven research is needed that enables scenario testing before the implementation and deployment of these tools. OBJECTIVE: This study's objectives are to (1) design an original, web-based vignette-based survey that features hypothetical scenarios based on emerging or real-world applications of AI-based CDS systems that will vary systematically by features related to clinical risk, the amount of information provided about the AI, and the AI result; and (2) test and determine causal effects of specific factors on the judgments and perceptions salient to physicians' clinical decision-making. METHODS: US-based physicians with specialties in family or internal medicine will be recruited through email and mail (target n=420). Through a web-based survey, participants will be randomized to a 3-part "sequential multiple assignment randomization trial (SMART) vignette" detailing a hypothetical clinical scenario involving an AI decision support tool. The SMART vignette design is similar to the SMART design but adapted to a survey design. Each respondent will be randomly assigned to 1 of the possible vignette variations of the factors we are testing at each stage, which include the level of clinical risk, the amount of information provided about the AI, and the certainty of the AI output. Respondents will be given questions regarding their hypothetical decision-making in response to the hypothetical scenarios. RESULTS: The study is currently in progress and data collection is anticipated to be completed in 2024. CONCLUSIONS: The web-based vignette study will provide information on how contextual factors such as clinical risk, the amount of information provided about an AI tool, and the AI result influence physicians' reactions to hypothetical scenarios that are based on emerging applications of AI in frontline health care settings. Our newly proposed "SMART vignette" design offers several benefits not afforded by the extensively used traditional vignette design, due to the 2 aforementioned features. These advantages are (1) increased validity of analyses targeted at understanding the impact of a factor on the decision outcome, given previous outcomes and other contextual factors; and (2) balanced sample sizes across groups. This study will generate a better understanding of physician decision-making within this context. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/54787.

Academic machine learning researchers' ethical perspectives on algorithm development for health care: a qualitative study.

OBJECTIVES: We set out to describe academic machine learning (ML) researchers' ethical considerations regarding the development of ML tools intended for use in clinical care. MATERIALS AND METHODS: We conducted in-depth, semistructured interviews with a sample of ML researchers in medicine (N = 10) as part of a larger study investigating stakeholders' ethical considerations in the translation of ML tools in medicine. We used a qualitative descriptive design, applying conventional qualitative content analysis in order to allow participant perspectives to emerge directly from the data. RESULTS: Every participant viewed their algorithm development work as holding ethical significance. While participants shared positive attitudes toward continued ML innovation, they described concerns related to data sampling and labeling (eg, limitations to mitigating bias; ensuring the validity and integrity of data), and algorithm training and testing (eg, selecting quantitative targets; assessing reproducibility). Participants perceived a need to increase interdisciplinary training across stakeholders and to envision more coordinated and embedded approaches to addressing ethics issues. DISCUSSION AND CONCLUSION: Participants described key areas where increased support for ethics may be needed; technical challenges affecting clinical acceptability; and standards related to scientific integrity, beneficence, and justice that may be higher in medicine compared to other industries engaged in ML innovation. Our results help shed light on the perspectives of ML researchers in medicine regarding the range of ethical issues they encounter or anticipate in their work, including areas where more attention may be needed to support the successful development and integration of medical ML tools.

Self-reported influences on willingness to receive COVID-19 vaccines among physically ill, mentally ill, and healthy individuals.

OBJECTIVE: Individuals with mental and physical disorders have been disproportionately affected by adverse health outcomes due to the COVID-19 pandemic, and yet vaccine hesitancy persists despite clear evidence of health benefits. Therefore, our study explored factors influencing willingness to receive a COVID-19 vaccine. METHODS: Individuals with mental illness (n = 332), physical illness (n = 331), and no health issues (n = 328) were recruited via Amazon Mechanical Turk. Participants rated willingness to obtain a fully approved COVID-19 vaccine or a vaccine approved only for experimental/emergency use and influences in six domains upon their views. We examined differences by health status. RESULTS: Participants across groups were moderately willing to receive a COVID-19 vaccine. Perceived risk was negatively associated with willingness. Participants differentiated between vaccine risk by approval stage and were less willing to receive an experimental vaccine. Individuals with mental illness rated risk of both vaccines similarly to healthy individuals. Individuals with physical illness expressed less willingness to receive an experimental vaccine. Domain influences differently affected willingness by health status as well as by vaccine approval status. CONCLUSIONS: Our findings are reassuring regarding the ability of people with mental disorders to appreciate risk in medical decision-making and the ability of people of varied health backgrounds to distinguish between the benefits and risks of clinical care and research, refuting the prevailing notions of psychiatric exceptionalism and therapeutic misconception. Our findings shine a light on potential paths forward to support vaccine acceptance.

Factors Influencing Perceived Helpfulness and Participation in Innovative Research: A Pilot Study of Individuals with and without Mood Symptoms.

Little is known about how individuals with and without mood disorders perceive the inherent risks and helpfulness of participating in innovative psychiatric research, or about the factors that influence their willingness to participate. We conducted an online survey with 80 individuals (self-reported mood disorder [n = 25], self-reported good health [n = 55]) recruited via MTurk. We assessed respondents' perceptions of risk and helpfulness in study vignettes associated with two innovative research projects (intravenous ketamine therapy and wearable devices), as well as their willingness to participate in these projects. Respondents with and without mood disorders perceived risk similarly across projects. Respondents with no mood disorders viewed both projects as more helpful to society than to research volunteers, while respondents with mood disorders viewed the projects as equally helpful to volunteers and society. Individuals with mood disorders perceived ketamine research, and the two projects on average, as more helpful to research volunteers than did individuals without mood disorders. Our findings add to a limited empirical literature on the perspectives of volunteers in innovative psychiatric research.

Influence of Dispositional Optimism on Ethically Salient Research Perspectives: A Pilot Study.

Research participants should be drawn as fairly as possible from the potential volunteer population. Underlying personality traits are underexplored as factors influencing research decision-making. Dispositional optimism, known to affect coping, physical health, and psychological well-being, has been minimally studied with respect to research-related attitudes. We conducted an exploratory, online survey with 151 individuals (with self-reported mental illness [n = 50], physical illness [n = 51], or neither [n = 50]) recruited via MTurk. We evaluated associations between dispositional optimism (assessed with the Life Orientation Test-Revised) and general research attitudes, perceived protectiveness of five research safeguards, and willingness to participate in research using safeguards. Strongly optimistic respondents expressed more positive research attitudes and perceived four safeguards as more positively influencing willingness to participate. Optimism was positively associated with expressed willingness to participate in clinical research. Our findings add to a limited literature on the influence of individual traits on ethically salient research perspectives.

Eating disorders early app use mediates treatment effect on clinical improvement.

OBJECTIVE: Eating disorders (EDs) contribute considerably to the global burden of disease. However, most affected individuals do not receive treatment. Mobile apps present an enormous opportunity to increase access to mental healthcare services. This study examined whether the degree of usage of a self-help app for EDs mediated the app's effects on the clinical response by individuals with EDs. METHOD: App usage measures included the total number of cognitive-behavioral meal logs, total number of days spent using the app, and the last day the app was used during the study period. Mediation analysis was performed using the MacArthur framework. RESULTS: All usage variables met the analytic requirements for testing mediation (group means (sd) for app and standard app, respectively: logs = 74 (108) vs. 51.4 (88.1), days spent = 14.3 (17.5) vs. 10.6 (15.0), p-values from Wilcox rank sum tests p < .01). Regression coefficients indicated mediation effects. The mediation effects demonstrated support that increased engagement (as measured by logs and time spent on the app) was related to an increased likelihood of achieving a significant clinical change by the end of the trial. DISCUSSION: Greater and longer engagement in an ED app mediates its efficacy in terms of ED remission.

Perceived protectiveness of research safeguards and influences on willingness to participate in research: A novel MTurk pilot study.

Little is known about how individuals with mood disorders view the protectiveness of research safeguards, and whether their views affect their willingness to participate in psychiatric research. We conducted an online survey with 80 individuals (self-reported mood disorder [n = 25], self-reported good health [n = 55]) recruited via MTurk. We assessed respondents' perceptions of the protectiveness of five common research safeguards, as well as their willingness to participate in research that incorporates each safeguard. Perceived protectiveness was strongly related to willingness to participate in research for four of the safeguards. Our findings add to a limited literature on the motivations and perspectives of key stakeholders in psychiatric research.

Artificial Intelligence and Suicide Prevention: A Systematic Review of Machine Learning Investigations.

Suicide is a leading cause of death that defies prediction and challenges prevention efforts worldwide. Artificial intelligence (AI) and machine learning (ML) have emerged as a means of investigating large datasets to enhance risk detection. A systematic review of ML investigations evaluating suicidal behaviors was conducted using PubMed/MEDLINE, PsychInfo, Web-of-Science, and EMBASE, employing search strings and MeSH terms relevant to suicide and AI. Databases were supplemented by hand-search techniques and Google Scholar. Inclusion criteria: (1) journal article, available in English, (2) original investigation, (3) employment of AI/ML, (4) evaluation of a suicide risk outcome. N = 594 records were identified based on abstract search, and 25 hand-searched reports. N = 461 reports remained after duplicates were removed, n = 316 were excluded after abstract screening. Of n = 149 full-text articles assessed for eligibility, n = 87 were included for quantitative synthesis, grouped according to suicide behavior outcome. Reports varied widely in methodology and outcomes. Results suggest high levels of risk classification accuracy (>90%) and Area Under the Curve (AUC) in the prediction of suicidal behaviors. We report key findings and central limitations in the use of AI/ML frameworks to guide additional research, which hold the potential to impact suicide on broad scale.

Attitudes of Mothers Regarding Willingness to Enroll Their Children in Research.

This study assessed mothers' perspectives regarding research involvement by their children, factors that might affect perceptions of research risks, and attitudes regarding willingness to enroll children in research. Participants completed a survey on Amazon Mechanical Turk. Mothers were less inclined to enroll children in research involving procedures posing higher risk (regression coefficient = -0.51). Mothers without mental health issues with children without health issues were more sensitive to risk than mothers without mental health issues with children with health issues (estimated difference = 0.49). Mothers with mental health issues were more willing than mothers without mental health issues to enroll children in research (regression coefficient = -0.90). Among mothers with mental health issues, having a child with a health issue was associated with increased willingness to enroll in research, compared with having children without health issues (estimated difference = 0.65).

Negative association of perceived risk and willingness to participate in innovative psychiatric research protocols.

Psychiatric researchers grapple with concerns that individuals with mental illness may be less likely to appreciate risks of research participation, particularly compared to people not suffering from mental illness. Therefore, empirical studies that directly compare the perspectives of such individuals are needed. In addition, it is important to evaluate perspectives regarding varied types of research protocols, particularly as innovative psychiatric research protocols emerge. In this pilot study, respondents with a mood disorder (n = 25) as well as respondents without a mood disorder (n = 55) were recruited using Amazon's Mechanical Turk (MTurk) platform. These respondents were surveyed regarding four psychiatric research projects (i.e., experimental medication [pill form]; non-invasive magnetic brain stimulation; experimental medication [intravenous infusion]; and implantation of a device in the brain). Regardless of health status, respondents rated the four research protocols as somewhat to highly risky. The brain-device implant protocol was seen as the most risky, while the magnetic brain stimulation project was viewed as "somewhat risky". Respondents, on average and regardless of health status, rated their willingness at or below "somewhat willing." Respondents were least willing to participate in the brain-device implant protocol, whereas they were "somewhat willing" to participate in the magnetic brain stimulation protocol. Trust in medical research was negatively associated with perceived risk of research protocols. Perceived risk was negatively associated with willingness to participate, even when adjusting for potential confounders, suggesting that attunement to risk crosses diagnostic, gender, and ethnic categories, and is more salient to research decision-making than trust in medical research and dispositional optimism. The findings of this study may offer reassurance about the underlying decision-making processes of individuals considering participation in innovative neuroscience studies.

Safeguarding Participants in Psychiatric Genetic Research: Perspectives of Key Stakeholder Groups.

Public trust in research depends on implementation of research protections. Genetic and psychiatric research may elicit "exceptionalism," the belief that these types of research deserve special protections. Genetic information has been viewed as different from other health information. Psychiatric research has been scrutinized based on concerns about the impact of psychiatric illness on individuals' abilities to make decisions. This study compared four stakeholder groups' attitudes toward research safeguards. Psychiatric genetic researchers and institutional review board chairs received structured surveys. Individuals with mental illness and family members participated in semistructured interviews. Paired sample t-tests were used to compare mean ratings of importance of safeguard procedures for genetic versus nongenetic research on physical versus mental illnesses. All groups provided higher ratings for the importance of safeguards for genetic research and for mental illness. Individuals with mental illness and family members rated the importance of safeguards more highly than researchers and chairs did. Results of generalized linear models showed significant effects of gender and ethnicity.

Attitudes Regarding Enrollment in a Genetic Research Project: An Informed Consent Simulation Study Comparing Views of People With Depression, Diabetes, and Neither Condition.

In this study, participants with a self-reported history of depression, diabetes, or no illness underwent a simulated informed consent process for a hypothetical genetic study related to depression or diabetes. Participants completed a survey assessing their perceived understanding of the research process, perceptions of its risks and benefits, their satisfaction with the informed consent process, and their readiness to make a hypothetical enrollment decision. All participants indicated strong readiness to make an enrollment decision regarding the research characterized in the simulation. Participants reported understanding the consent process relatively well and being generally satisfied with it. Greater concerns were expressed regarding psychosocial risks than biological risks for genetic studies on mental disorders. Our study documented positive attitudes toward volunteering for research that involved the collection of genetic data.

Do human subject safeguards matter to potential participants in psychiatric genetic research?

Despite longstanding concerns about the adequacy of human research protections in mental illness investigations, minimal work has focused on the perspectives of key stakeholders regarding these safeguards. This investigation examined the perspectives of potential research participants regarding safeguards for psychiatric genetic research. Individuals with mental illness (n = 71), first-degree family members of individuals with mental illness (n = 54), and individuals with no personal or close family history of mental illness (n = 57) provided responses to items regarding perceptions of: 1) protectiveness of a range of research safeguards in genetic research on mental illness; 2) influence of these safeguards on research participation decision-making; and 3) importance of these safeguards depending on the nature of the research (i.e., genetic vs. non-genetic mental illness research; and genetic research on mental illness vs. physical illness). Potential research participants perceived existing safeguard procedures as generally protective. The three groups did not differ in their ratings of protectiveness, with the exception of the safeguard domain of "Informed Consent or Alternative Decision-Making Procedures," which was viewed as more protective by family members of people with mental illness than by individuals with mental illness or comparison participants. Safeguard procedures were perceived as strongly influential with respect to willingness to enroll in psychiatric genetic research. These findings suggest that the presence of safeguards positively influences enrollment decision-making by research volunteers and indicate that potential psychiatric genetic research participants find safeguards to be protective, underscoring the responsibility to implement safeguard practices conscientiously.

The Utility and Acceptability of a Self-Help Smartphone Application for Eating Disorder Behaviors.

OBJECTIVE: Smartphone applications (apps) for eating disorders are a promising approach to assist individuals who do not receive traditional treatment. This study examines usage characteristics, perceptions regarding the acceptability of a new self-help intervention developed for users with eating disorders, and associations between attitudes and use patterns. METHODS: 189 individuals pilot-tested a personalized app-based program, and 133 completed the required components of the pilot-test over an 8-day period. Of these, 64 individuals (49%) completed an exit survey pertaining to acceptability. RESULTS: Seventy percent of those who pilot-tested the app-based program completed the required components, i.e. a baseline review and then a 1-week assessment. Body mass index was associated with the total number of recorded meal logs. Study participants rated the app as highly suitable and acceptable, providing evidence of the feasibility and appropriateness of the program. CONCLUSION: The app-based program demonstrated feasibility of deploying the app across user groups and high acceptability.

Views of the importance of psychiatric genetic research by potential volunteers from stakeholder groups.

Few studies have explored potential volunteers' attitudes toward genetic research. To address this gap in the literature, we developed an empirical project to document views held by individuals who may wish to enroll in genetic studies involving mental disorders. People living with mental illness, family members of people with mental illness, and community comparison volunteers were queried regarding their views on the importance of genetic research generally, in comparison with medical research, and in relation to 12 health conditions categorized in four types. T-tests and univariate and multivariate analysis of variance were used as appropriate. Participants expressed support for the importance of genetic research (mean = 9.43, scale = 1-10) and endorsed genetic research more highly compared with non-genetic medical research (mean = 9.43 vs. 8.69, P value = <0.001). The most highly endorsed genetic research was for cognitive disorders, followed by mental illness disorders, physical illness disorders, and addiction disorders (means = 8.88, 8.26, 8.16 and 7.55, respectively, P value = <0.001). Overall, this study provides evidence of strong endorsement of genetic research over non-genetic research by potential volunteers.

Perspectives of psychiatric investigators and IRB chairs regarding benefits of psychiatric genetics research.

There is hope that psychiatric genetics inquiry will provide important insights into the origins and treatment of mental illness given the burden of these conditions. We sought to examine perspectives of psychiatric genetic investigators regarding the potential benefits of genetic research in general and the potential benefits of genetic research for the diagnosis and treatment of mental illnesses specifically. We compared investigator attitudes with those of chairs of Institutional Review Boards (IRBs) entrusted with evaluating the benefits and risks of human research studies. Two groups directly engaged with the conduct and oversight of psychiatric genetic research were examined (psychiatric investigators, n = 203; IRB Chairs, n = 183). Participants rated 15 survey items regarding current and future benefits of general genetic research, possible benefits of psychiatric genetic research, and the importance to society of genetic vs. non-genetic research examining causes and treatments of illnesses. Investigators and IRB Chairs strongly endorsed the future benefits of general genetic research for society and for the health of individuals; compared to IRB Chairs, investigators were more positive about these benefits. Even after adjusting for demographic variables, psychiatric genetic investigators were significantly more optimistic about genetic research compared with IRB Chairs. Both groups were moderately optimistic about the possible benefits of genetic research related to mental illness. Greater optimism was seen regarding new or personalized medications for mental illnesses, as well as genetic predictive testing of mental illnesses. Greater precision and circumspection about the potential benefits of psychiatric genetic research are needed.

Does informed consent given by healthy individuals when enrolling in clinical research feel less voluntary than for ill individuals?

Clinical research is predicated ethically on the authentic voluntarism of individuals who choose to enroll in human studies. Existing literature has focused on aspects of informed consent for clinical research other than voluntarism. The objective of this study was to compare the perspectives of clinical research participants who are in good health and who are ill regarding voluntarism-related aspects of informed consent and to assess situational influences that enable voluntarism in the process of obtaining clinical research consent. A 23-item written survey, the Informed Consent Questionnaire (ICQ), was administered in a "piggyback" semi-structured interview study of ill and healthy volunteers enrolled in IRB-approved clinical research studies. A total of 150 (60 mentally ill, 43 physically ill, and 47 healthy) clinical research participants participated. Respondents expressed positive views of their experiences with the informed consent processes for their respective clinical research protocols and respondents strongly endorsed items related to voluntarism irrespective of their illness type (range of means = [3.9, 4.8]). Ill participants more highly endorsed items relating to informed consent conditions (mentally ill vs healthy: 0.54 on a 5-point scale, P value = 0.01) (physically ill vs. healthy: 0.47 on a 5-point scale, P value = 0.01). The favorable views of clinical research participants regarding their experience of giving informed consent to enroll in a study were not surprising. Contrary to our a priori hypothesis, healthy individuals did not feel as positively as their ill counterparts.

Subgrouping the users of a specialized app for eating disorders.

Although mobile technologies for eating disorders (EDs) are burgeoning, there is limited data about the clinical characteristics of individuals using specialized smartphone applications (apps) without accompanying traditional forms of treatment. This study evaluated whether the users of an ED app cluster in clinically meaningful groups. Participants were 1,280 app users (91.3% female; mean age 27) who reported not being in a weekly treatment for their ED. A hierarchical cluster analysis distinguished five groups of participants, all approximating DSM-5 ED categories. One cluster comprised of non-female, ethnically diverse users with Bulimia Nervosa features. Findings suggest that app users resemble known patient classifications.